American Sign Language
Learning Game
Inspiration
After working retail for a year, I started to realize that not all customers were able to be served equally. Customers with hearing impairments are often forced to find creative ways to communicate to customer service associates due to the lack of familiarity with ASL. This often stressful and time consuming for both the customer and the employee. I wanted to make this game as a fun "Hello World" to learning sign language through gesture recognition programming. This program would assist associates to learn sign language to better serve their customers.
How It Works
The game is written in Python and uses the hand tracking feature developed by mediapipe. To make the game features easier to write I also used pygame and openCV to control camera feeds. Once the game is started, the player is led through a Demo mode. This shows the player a picture of the correct gesture for the color to be signed. The player is then to hold their left hand in front of the camera and mimic the gesture image. Using the hand tracking, the program is able to determine if the person's fingers and thumb are in the correct orientation.
Once the correct gesture is achieved for 3 seconds, the user is given a green check mark and the corresponding color is displayed. After the demo mode is complete, there is a challenge mode that allows the user to test their knowledge. The prompt image is taken away and the user is given a random color to sign. To win the game the user has to get five points.
Demo Mode
Challenge Mode
